This reproduces a bug reported multiple times over the years in sparkql evaluation. We held off on a fix as it was clear the implementation needed a rewrite, and would be trivial in the v2 AST implementation we'd like to go in.

Well, that v2 branch is getting stale, and this bug has popped up again on CUR-29245, so I opted to split the difference in a rewrite that's closer to how a tree solution would operate.

Processing the expressions in the same block group is fairly trivial in the old implementation. The nested expressions is where things get tricky in the old implementation. Therefore, this implementation breaks up and groups a filter into expressions of the same block group.

From there, I process each block group starting with the most deeply nested groups. As we go back up the nesting levels, the block group is reduced to a result and a conjunction, which slides in like a placeholder for and expression. Thus the block group becomes a faux expression in the block level above it.

An AST approach would still be superior here, but the performance differences should be negligible.

Example:

# Original filter:
BlockGroup1A Eq true And BlockGroup1B Eq True And (BlockGroup2X Eq false Or BlockGroup2Y Eq false)

# block group one:
BlockGroup1A Eq true And BlockGroup1B Eq True

# block group two:
BlockGroup2X Eq false Or BlockGroup2Y Eq false

# First iteration:
BlockGroup1A Eq true And BlockGroup1B Eq True And (true)

# second iteration
true

# Finalize
true

Final thoughts: we got away with an ugly implementation for a long time, as the typical saved searches were fairly basic. As our UI and such started supporting more complex queries, we need to keep up.